Understanding and designing embodied experiences through mid-air tactile stimulation

Current attempts to render touch in multimedia technology still represents a challenge. Touch is indeed a complex system and there are many aspects to take into account when trying to rendering it (e.g. the compliance of an object, its weight, orientation, geometric properties and forces on the skin). This is especially true for VR, where touch is an important factor to achieve the embodiment in virtual environments. Recently, new tactile technology has been developed: the mid-air devices, capable of delivering tactile feedback without entering in contact with the skin. One of the contributions of the doctoral research described in this paper is to overcome design challenges and create immersive experiences by applying psychological principles and paradigms, exploiting the advantages of the mid-air technology. We designed possible embodied interaction scenarios in mid-air and physical touch. Findings from these research point to opportunities for designing new immersive experiences. Future work will involve different parts of the body and different tactile properties (e.g. thermal stimulation)

Touching is believing: creating illusions and feeling of embodiment with mid-air haptic technology

Over the last two decades, the sense of touch has received new attention from the scientific community.Several haptic devices have been developed to address the complexity of the sense of touch, the latest addition being mid-air (contactless) haptic technology. An interesting series of previous research has suggested an easier way to tackle the complexity of designing convincing tactile sensations by exploiting tactile illusions. Tactile illusions rely on perceptual shortcuts based on the psychophysics of the tactile receptors. Currently, studies exploring the perceptual space of mid-air haptics and its applicability in the tactile illusions field are still limited in number. This thesis aims to contribute to the field of Human-Computer Interaction (HCI) by investigating the perceptual design space of ultrasonic mid-air haptics technology. Specifically, in a first set of three studies, we investigate the absolute thresholds (minimal amount of a property of astimulus that a user can detect) for control points (CP) at different frequencies on the hand and arm (Study 1). Then we investigate the optimal sampling rate needed to drive the device in an optimal fashion and its relationship with shape size (Study 2). Next, we apply a new technique to increase users’ performance in a shape discrimination task (Study 3). In Study 4, we start the exploration of a tactile illusion of movement using contact touch and later, we apply a similar procedure to investigate the feasibility of creating a tactile illusion of movement between the two non-interconnected hands by using mid-air touch (Study 5). Finally, in Study 6, we explore our sense of touch in VR, while providing an illusion of rain drops through mid-air haptics, to recreate a virtual hand illusion (VHI) to explore the boundaries of our sense of embodiment. Therefore, the contribution of this work is threefold: a) we contribute by adding new knowledge on the psychophysical space for mid-air haptics, b) we test the potential to create realistic tactile sensations by exploiting tactile illusions with mid-air haptic technology, and c) we demonstrate how tactile illusions mediated by mid-air haptics can convey a sense of embodiment in VR environments

I’m sensing in the rain: spatial incongruity in visual-tactile mid-air stimulation can elicit ownership in VR users

Major virtual reality (VR) companies are trying to enhance the sense of immersion in virtual environments by implementing haptic feedback in their systems (e.g., Oculus Touch). It is known that tactile stimulation adds realism to a virtual environment. In addition, when users are not limited by wearing any attachments (e.g., gloves), it is even possible to create more immersive experiences. Mid-air haptic technology provides contactless haptic feedback and offers the potential for creating such immersive VR experiences. However, one of the limitations of mid-air haptics resides in the need for freehand tracking systems (e.g., Leap Motion) to deliver tactile feedback to the user's hand. These tracking systems are not accurate, limiting designers capability of delivering spatially precise tactile stimulation. Here, we investigated an alternative way to convey incongruent visual-tactile stimulation that can be used to create the illusion of a congruent visual-tactile experience, while participants experience the phenomenon of the rubber hand illusion in VR

Mid-air haptic rendering of 2D geometric shapes with a dynamic tactile pointer

An important challenge that affects ultrasonic midair haptics, in contrast to physical touch, is that we lose certain exploratory procedures such as contour following. This makes the task of perceiving geometric properties and shape identification more difficult. Meanwhile, the growing interest in mid-air haptics and their application to various new areas requires an improved understanding of how we perceive specific haptic stimuli, such as icons and control dials in mid-air. We address this challenge by investigating static and dynamic methods of displaying 2D geometric shapes in mid-air. We display a circle, a square, and a triangle, in either a static or dynamic condition, using ultrasonic mid-air haptics. In the static condition, the shapes are presented as a full outline in mid-air, while in the dynamic condition, a tactile pointer is moved around the perimeter of the shapes. We measure participants’ accuracy and confidence of identifying shapes in two controlled experiments (n1 = 34, n2 = 25). Results reveal that in the dynamic condition people recognise shapes significantly more accurately, and with higher confidence. We also find that representing polygons as a set of individually drawn haptic strokes, with a short pause at the corners, drastically enhances shape recognition accuracy. Our research supports the design of mid-air haptic user interfaces in application scenarios such as in-car interactions or assistive technology in education

I see where this is going: a psychophysical study of directional mid-air haptics and apparent tactile motion

Mid-air haptic technology can render a plethora of tactile sensations including points, lines, shapes, and textures. To do so, one requires increasingly complex haptic displays. Meanwhile, tactile illusions have had widespread success in the development of contact and wearable haptic displays. In this paper, we exploit the apparent tactile motion illusion to display mid-air haptic directional lines; a prerequisite for the rendering of shapes and icons. We present two pilot studies and a psychophysical study that contrasts a dynamic tactile pointer (DTP) to an apparent tactile pointer (ATP) in terms of direction recognition. To that end, we identify optimal duration and direction parameters for both DTP and ATP mid-air haptic lines and discuss the implications of our findings with respect to haptic feedback design, and device complexity

Creating an illusion of movement between the hands using mid-air touch

Apparent tactile motion (ATM) has been shown to occur across many contiguous parts of the body, such as fingers, forearms and the back. More recently, the illusion has also been elicited on non-contiguous part of the body, such as from one hand to the other when interconnected or not interconnected by an object in between the hands. Here we explore the reproducibility of the intermanual tactile illusion of movement between two free hands by employing mid-air tactile stimulation. We investigate the optimal parameters to generate a continuous and smooth motion using two arrays of ultrasound speakers, and two stimulation techniques (i.e. static vs. dynamic focal point). In the first experiment, we investigate the occurrence of the illusion when using a static focal point, and we define a perceptive model. In the second experiment, we examine the illusion using a dynamic focal point, defining a second perceptive model. Finally, we discuss the differences between the two techniques

Haptic rules! augmenting the gaming experience in traditional games: the case of Foosball

Haptic sensations are a crucial aspect of everyday interaction. We touch, lift, move, and probe objects in our everyday activities. However, whilst the importance of haptic feedback has often been emphasized in gaming, haptics has been rarely used to enhance the experience in traditional (non-digital) games. In the last 50 years, technological advancement has allowed an easier access to haptic feedback. Digital games have exploited such access mainly (1) to enhance visual and acoustic feedback, and (2) to reproduce realistic feedbacks in augmented and virtual environments. Here, we re-think haptic feedback by focusing on game augmentation to enrich the gaming experience and gameplay in non-technological games. We describe the design process that led us to define the concept of “haptic rules” as haptic-based enhancement in interference play, where haptic feedback is delivered by users to users within the game as a further mode of interaction. We apply the idea of haptic rules to the game of foosball, evaluating the effect on the gameplay and user experience

Sampling strategy for ultrasonic mid-air haptics

Mid-air tactile stimulation using ultrasonics has been used in a variety of human computer interfaces in the form of prototypes as well as products. When generating these tactile patterns with mid-air tactile ultrasonic displays, the common approach has been to sample the patterns using the hardware update rate capabilities to their full extent. In the current study we show that the hardware update rate can impact perception, but unexpectedly we find that higher update rates do not improve pattern perception. In a first user study, we highlight the effect of update rate on the perceived strength of a pattern, especially for patterns rendered at slow rate of less than 10 Hz. In a second user study, we identify the evolution of the optimal update rate according to variations in pattern size. Our main results show that update rate should be designated as additional parameter for tactile patterns. We also discuss how the relationships we defined in the current study can be implemented into designer tools so that designers remain oblivious to this additional complexity

It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor Behaviour

Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this paper, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's (1) explicit judgment of the texture attributes, (2) explicit sensations of their own hand, and (3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse

Corrigendum: May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality

In the published article, there was an error regarding the affiliation(s) for Dario Pittera. Instead of having affiliation(s) ∗∗2,3,4∗∗, they should only have ∗∗2,4∗∗. The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated